Efficient Multiprogramming for Multicores with SCAF

As hardware becomes increasingly parallel and the availability of scalable parallel software improves, the problem of managing multiple multi-threaded applications (processes) becomes important. Malleable processes, which can vary the number of threads used as they run, enable sophisticated and flexible resource management. Although many existing applications parallelized for SMPs with parallel runtimes are in fact already malleable, deployed runtime environments provide no interface nor any strategy for intelligently allocating hardware threads or even preventing oversubscription. Work up until SCAF either depends upon profiling applications ahead of time in order to make good decisions about allocations, or does not account for process efficiency at all. This paper presents the Scheduling and Allocation with Feedback (SCAF) system, a drop-in runtime solution which supports existing malleable applications in making intelligent allocation decisions based on observed efficiency without any paradigm change, changes to semantics, program modification, offline profiling, or even recompilation. Our existing implementation can control most unmodified OpenMP applications. Other malleable threading libraries can also easily be supported with small modifications, without requiring application modification. In this work, we present the SCAF daemon and a SCAF-aware port of the GNU OpenMP runtime. We demonstrate that applications running on the SCAF runtime still perform well when executing on a quiescent system. We present a new technique for estimating process efficiency purely at runtime, and demonstrate that it produces consistent results. We show that the overhead of SCAF is negligible. In one measured multi-process scenario, the system throughput (as measured by sum of speedups) improved by nearly 3X using SCAF compared to using unmodified OpenMP libraries. Furthermore, in the same scenario, SCAF improved throughput by 26% versus equipartitioning. Finally, SCAF is able to equal the throughput of a system that uses a priori profiling, but without any profiling or other user involvement.